Perception Score: A Learned Metric for Open-ended Text Generation Evaluation

Automatic evaluation for open-ended natural language generation tasks remains a challenge. We propose learned metric: Perception Score. It utilizes pre-trained model and considers context information conditional generation. Score assigns holistic score along with the uncertainty measurement. conduct experiments on three two unconditional tasks. achieves state-of-the-art results all consistently in terms of correlation human scores.